Statistical methods for the comparison of stochastic optimizers

The comparison of stochastic algorithms for optimization requires the use of appropriate statistical methods for the analysis of results. Commonly used parametric tests, like the t-test or ANOVA procedures in experimental design, are based on the assumption that algorithms’ results are normally distributed. However, this assumption is often violated because the distribution of the cost values of the final solutions is commonly very asymmetric. For example, in the case of minimization problems the distribution is likely to be bounded on the left and skewed to the right.